Existing graph neural networks may suffer from the "suspended animation problem" when the model architecture goes deep. Meanwhile, for some graph learning scenarios, e.g., nodes with text/image attributes or graphs with long-distance node correlations, deep graph neural networks will be necessary for effective graph representation learning. In this paper, we propose a new graph neural network, namely DIFNET (Graph Diffusive Neural Network), for graph representation learning and node classification. DIFNET utilizes both neural gates and graph residual learning for node hidden state modeling, and includes an attention mechanism for node neighborhood information diffusion. Extensive experiments will be done in this paper to compare DIFNET against several state-of-the-art graph neural network models. The experimental results can illustrate both the learning performance advantages and effectiveness of DIFNET, especially in addressing the "suspended animation problem".

The existing graph neural networks (GNNs) based on the spectral graph convolu-tional operator have been criticized for its performance degradation, which is especially common for the models with deep architectures. In this paper, we further identify the suspended animation problem with the existing GNNs. Such a problem happens when the model depth reaches the suspended animation limit, and the model will not respond to the training data any more and become not learn-able. Analysis about the causes of the suspended animation problem with existing GNNs will be provided in this paper, whereas several other peripheral factors that will impact the problem will be reported as well. Different from the other learning settings, the extensive connections in the graph data will render the existing simple residual learning methods fail to work. We prove the effectiveness of the introduced new graph residual terms from the norm preservation perspective, which will help avoid dramatic changes to the node's representations between sequential layers. Graph neural networks (GNN), e.g., graph convolutional network (GCN) Kipf & Welling (2016) and graph attention network (GAT) V eli ˇ ckovi c et al. (2018), based on the approximated spectral graph convolutional operator Hammond et al. (2011), can learn the representations of the graph data effectively. Meanwhile, such GNNs have also received lots of criticism, since as these GNNs' architectures go deep, the models' performance will get degraded, which is similar to observations on other deep models (e.g., convolutional neural network) as reported in He et al. (2015). Meanwhile, different from the existing deep models, when the GNN model depth reaches a certain limit (e.g., depth 5 for GCN with the bias term disabled or depth 8 for GCN with the bias term enabled on the Cora dataset), the model will not respond to the training data any more and become not learn-able.